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The problem of the universal compression of a sequence from a library of several small to moderate length sequences from 
similar context arises in many practical scenarios, such as the compression of the storage data and the Internet traffic. In such 
scenarios, it is often required to compress and decompress every sequence individually. However, the universal compression 
of the individual sequences suffers from significant redundancy overhead [1], [2 |. In this paper, we aim at answering whether 
£Nl or not having a memory unit in the middle can result in a fundamental gain in the universal compression. We present the 
problem setup in the most basic scenario consisting of a server node S, a relay node R (i.e., the memory unit), and a client 
node C, as depicted in Fig Q] We assume that server S wishes to send the sequence x n to the client C who has never had 
any prior communication with the server, and hence, is not capable of memorization of the source context. However, R has 
previously communicated with S to forward previous sequences from S to the clients other than C (not shown in Fig. [T]), 
and thus, R has memorized a context y m shared with S. Note 

that if the relay node was absent the source could possibly apply M \ ( f ^ 

universal compression to x n and transmit to C whereas the presence 
of memorized context at R can possibly reduce the communication 

overhead in S-R link. Pig- !• The basic memory-assisted compression scenario. 

We let the information source located in S be parametric with <i-dimensional parameter vector 9 that is unknown a priori. 
Let l n denote the universal length function for the compression of the sequence x n . Denote l n \ m as the universal length 
function for x n with context memorization, where the encoder (at S) and the decoder (at R) have access to a memorized 
context y m . Our goal is to investigate the fundamental gain of the context memorization in the memory-assisted universal 
compression of the sequence x n over conventional universal source coding. To do so, we compare two schemes: 
■ • Ucomp (Universal compression of an individual sequence with no context memorization), in which a sole universal 
0^ , compression is applied on the sequence x n without context memorization. 

• UcompM (Universal compression of an individual sequences with context memorization), in which the encoder and the 
decoder will both have access to the memorized context y m from the same source, and they utilize y m as the context 
for the compression of the sequence x n . 
^ , Let H n (9) be the source entropy given 9. Further, let Q(l n , l n \mi &) — M^Jx™) » where E denotes expectation with respect 
to the true unknown parameter 8. Let e be such that < e < 1. We denote g(n, m, e) as the fundamental gain of the context 
t— I memorization (relative to the conventional universal compression without memory) for a sequence of length n from family 
| V of parametric sources using a memory of length m for a fraction 1 — e of the sources. That is 

g(n, m, e) = sup {z : P [Q(l n , l n \m> &) > z ] > 1 ~ e } • 
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C3 ' The following theorem presents a lower bound on the gain of memory-assisted source coding. 

Theorem 1 Assume that the parameter vector 9 follows Jeffreys' prior in the universal compression of the family of 
parametric sources V . Then, 

t x ^ , . Rn + log(e) - R(n, m) ( 1 

o(n, m, e) > H h u ' 

H n (8)+R(n,m) 

where R n is the average minimax redundancy, and R(n, rn) = ^ log (l + ^) + 2. 

Our result shows that UcompM achieves more than 50% memorization gain as compared to Ucomp in the compression of 
a first-order Markov sequence of length n = 128kB with a memory of size m = 8MB. Further, with sufficient memory, x n 
can be compressed to the enttopy rate. 
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